MacHome 2001 January/MacHome Magazine Demo Disc January 2001.iso/pc/Software/Applications/Squeak app design envir. ….sea/Squeak app design envir. 2.8/README.MacOT

Text File | 2000-06-18 | 9.8 KB | 195 lines | [TEXT/R*ch]

Notes from John M McIntosh <[e-mail]>
Subject: New Mac VM 68K and Open Transport (Q/A) PPC too.



PPC Notes:

For the PPC you need Open Transport 1.1.2, why? Well earlier versions are
broken. If you have an earlier version you can find a 1.1.2 updater on Apple
support site. The new VM was tested on PPC version of the Mac OS from
version 7.5.3 thru to OS 9.04 (7.5.3 7.5.3 update 2, 7.5.5 7.6, 7.6.1 8.0
8.1 8.5 8.6 9.0x) (Sigh did I miss one)? Also with various flavors of OT.
Having a test suite made this task tedious but viable.

68K Notes:

One of the goals when writing the new open transport based tcp/ip code for
the macintosh was that it really needed to run on 68K machines. This support
is NOT shipped with the standard VM, when running on a 68K machine you run
the old MacTCP version. Perhaps later we'll offer the OT version as a
plugin, here's why:

The Fat VM is built using Apple's code fragment manager (CFM). This means
for one thing you must have the 68K version of the CFM manager on your 68K
machine, since bugs exist in some implementations ensure you have the latest
from Apple's support site. Of course in order to run Squeak you need this...

The issue! Apple in their wisdom introduced a CFM version of OT for version
1.3. The problem is that OT 1.3 is NOT a standalone upgrade, it comes with
System 8.1. Therefore the OT version of the 68K code only would work on a
limited number of 68040 machines and you need a System 8.0 cd and the 8.1
updater. Now if you do have a version of OT 1.3 around you can copy the
libraries and use them on earlier OS back to 7.x. One of my test machines is
a lowly 68030 Mac SE/30 which used 7.5.5 and OT 1.3 to prove this point.

There may or may not be an improvement gain, the macTCP software has a
shorter code path than the OT code, but the new code runs faster if you
increase the buffers you hand the primitives and the listen/accept logic
doesn't drop any incoming connections so it's more robust, it took a few
hours to run the test suite but it did run. Oh and there's the UDP support.

For people that really want a 68K version with OT you can email me and I'll
see about shipping you one, at some point here I guess we'll post a real
version once I've seen any bug reports. For developers who have Code Warrior
around and time on your hands you can build your own.

Lastly I don't have permission to give anyone the OT1.3 libraries, sorry
don't ask...


Subject: Socket Test Suite (is coming)


As soon as I get the test suite to work on Linux I'll ship it.

When I was building the macintosh Open Transport code an interesting
objective was given. You must mimic the behavior of the old MacTCP code.
Well how and how do you test that? Of course the code is only documentation
it's not if there is a specification around. Oh and make it work on 68K and
PPC and all versions of the Mac OS back till the pre-Jobs era when we had
the OS flavor of the month, that just screams for automation eh?

So to solve this problem and preserve my sanity I bit the bullet and spent
many many hours building SUnit test case(s). Once I ship them you'll see
they are rough but the important thing was that I can run them on an old VM
using macTCP and the new VM using OT and they agree! They also allowed me to
tinker somewhat with the Smalltalk Socket class and ensured that the
behavior was the same as the historical pattern, well almost nope 257.0.0.0
is NOT a valid IP address, previous version of the resolver code disagreed
on this point.

To the people that promote this as the way to go, yes I coded test cases
then wrote the C code. I agree this is great, many times I found I wrote
'perfect' C code then tripped over a failure cheerfully pointed out by the
test suite. Also a few problems with OT versions popped up when I ran the
suite across a zillion macintosh OSs. In retrospect attempting to prove that
the code would work across all these different macintosh environments would
have proved impossible without a test suite.

I am hoping here that this will lay the foundation for a test suite we can
all use to validate the behavior of the socket primitives across all the
platform implementations. Right now there are differences between how the
mac version behaves and the linux version works, these little gothas can
cause problems for application programmers. Of course this should also
expand to cover other important plugins that are platform dependent.

Oh one last little thing I managed to violate an XP tenet and shipped Dan a
version of code to fix a very interesting problem he discovered at the last
minute and he promptly shipped that code to you all. Then I ran the full
test suite and discovered to my horror my one line change caused dreadful
side effects if you do socket close;close;close  (Not that you do that in
production code). This of course prompted a fix which is in the update
stream, the VM on the server was fixed early early this morning, and I don't
think anyone got their hands on the broken one.


Subject: Mac VM and Open Transport and performance numbers.


For the curious one of the things I attempted to do was to support a high
performance Squeak based tcp/ip server when I was writing the new software.
This drove some design decisions in the new C code. Once the code was
written, well how to test, since in reality I didn't really have a high
performance tcp/ip server to use, the intent is there, just no code.

To solve this issue I wrote a little application that builds a listen socket
and spins off a socket to handle the incoming request, which picks a
solution from a collection of random choices (think static HTML) and sends
that to the client and then closes the socket, much like many HTTP servers
do today. Multiple Smalltalk processes and shared queues are used to do
this. (I'll ship this application in a day or so).

I was interested in how many connections per second, how big was the backlog
queue, how much data was shipped, how long did servicing each request take,
and did we drop anything.

I ran this application on a PowerBook 500Mhz and used various clients on a
10Mbit ethernet segment to continuously make requests to it.

Ok the numbers:

I was limited to the number of clients but I served up 48 connections per
second sustained that's about 4.1 millions connects per day. Peak was 112
connections per second, with 21 requests outstanding at anyone point on
average. One of my tests moved 12.59 GB of data, or about 3.815 Mbits a
second with an average of 9,887 bytes per response (A few bytes to 20,000
were the choices). I had some other test results where I used some more
clients and got 58 connections per second (that's 5 million per day) with a
average bit rate of 4,706,028. One case using a 100MBit Lan did over 12MBit
peak. You should be aware that I could browse around in the server image
while this was going on and I didn't have any response time issues.

Service request times.
I took a random sample of 25945 connections across 7 hours and found:
75% completed in 100ms That is the time from the point of accept at the
listen to when we have sent the data and closed the socket.

84.36% completed in 500ms or less
89.63% completed in 1000ms or less
95.51% completed in 2000ms or less
98.05% completed in 5000ms or less
99.16% completed in 8000ms or less
99.65% completed in 10 seconds or less
with 1 at 17 and 1 at 24 seconds.

Please note the client software actually forked off 7 connections per
iteration thru a loop and I think some of the clients were CPU starved and
some buffer flow logic in the code tends to skew to higher numbers, but the
waitForDataUntil was set to 10 seconds, so nearly all the clients were
serviced within one iteration of their read loop.

In a 7 hour run I did not drop any data according to the client logs, and I
had one read failure, one of my Linux VMs locked up! Since I don't have
anything to compare this to I can't say how good it is. Mind it would appear
a midrange Squeak server could saturated a T1 line.

Oh and the interesting thing was this testing pointed out a number of issues
with the external semaphore signaling table which of course is fixed in the
latest updates.

Subject: Re: Squeak, NT, CGI and Microsofts web server?
on 6/17/00 4:18 AM, Stefan Matthias Aust at [e-mail] wrote:

> Probably not with a default Squeak.  I haven't tested but I think I read
> something like that socket requests are serialized because the VM can't
> deal with multiple simultaneous request.  Or perhaps that's only true for a
> Mac?
>

On the mac this is no longer true, the new OT socket code does most things
at the interrupt level. In testing I had as much as 150 incoming socket
accepts pending. Technically each of these had been accepted at the tcp/ip
level and were waiting for Squeak to confirm the accept or disconnect (of
course we wanted to accept them).

The same applies for reading, incoming data is of course queued at the
socket level, but at some 'magical' point based on socket buffer usage and
thresholds etc etc it notifies the application that data has arrived via an
interrupt, you can of course fiddle with these thresholds using  socket
options if you dare. As mentioned at some point we get an interrupt from the
OS and we read the data into some statically allocated buffers and signal
the read semaphore if required. Later when Squeak gets around to reading the
data, we of course just copy data out of the buffer into the supplied
Smalltalk string and put the buffer back on a free queue.


Note one of the interesting things here is what if the statically allocated
buffers are exhausted. At this point by not handling the read interrupt it
causes the invocation of tcp/ip flow control, and we also signal we have
data. Later when Squeak wakes up we read data from the socket directly into
the Squeak string. At some point we hope the buffers will be freed and go
back to normal processing. This logic prevents all work from stopping.